Sequence-to-Sequence Video Captioning with Residual Connected Gated Recurrent Units
نویسندگان
چکیده
Recurrent neural networks have recently emerged as a useful tool in computer vision and language modeling tasks such image video captioning. The main limitation of these is preserving the gradient flow network gets deeper. We propose captioning approach that utilizes residual connections to overcome this maintain by carrying information through layers from bottom top with additive features. experimental evaluations on MSVD dataset indicate proposed achieves accurate caption generation compared state-of-the-art results. In addition, integrated our custom-designed Android application, WeCapV2, capable generating captions without an internet connection.
منابع مشابه
Sequence to Sequence Model for Video Captioning
Automatically generating video captions with natural language remains a challenge for both the field of nature language processing and computer vision. Recurrent Neural Networks (RNNs), which models sequence dynamics, has proved to be effective in visual interpretation. Based on a recent sequence to sequence model for video captioning, which is designed to learn the temporal structure of the se...
متن کاملConsensus-based Sequence Training for Video Captioning
Captioning models are typically trained using the crossentropy loss. However, their performance is evaluated on other metrics designed to better correlate with human assessments. Recently, it has been shown that reinforcement learning (RL) can directly optimize these metrics in tasks such as captioning. However, this is computationally costly and requires specifying a baseline reward at each st...
متن کاملSequence Modeling using Gated Recurrent Neural Networks
In this paper, we have used Recurrent Neural Networks to capture and model human motion data and generate motions by prediction of the next immediate data point at each time-step. Our RNN is armed with recently proposed Gated Recurrent Units which has shown promissing results in some sequence modeling problems such as Machine Translation and Speech Synthesis. We demonstrate that this model is a...
متن کاملRRA: Recurrent Residual Attention for Sequence Learning
In this paper, we propose a recurrent neural network (RNN) with residual attention (RRA) to learn long-range dependencies from sequential data. We propose to add residual connections across timesteps to RNN, which explicitly enhances the interaction between current state and hidden states that are several timesteps apart. This also allows training errors to be directly back-propagated through r...
متن کاملRecurrent Residual Learning for Sequence Classification
In this paper, we explore the possibility of leveraging Residual Networks (ResNet), a powerful structure in constructing extremely deep neural network for image understanding, to improve recurrent neural networks (RNN) for modeling sequential data. We show that for sequence classification tasks, incorporating residual connections into recurrent structures yields similar accuracy to Long Short T...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Europan journal of science and technology
سال: 2022
ISSN: ['2148-2683']
DOI: https://doi.org/10.31590/ejosat.1071835